Papers with dialogue systems
Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems (2022.aacl-srw)
Copied to clipboard
| Challenge: | Existing methods for dialogue system evaluation are inefficient and time-consuming. |
| Approach: | They propose a dialogue collection method for automating dialogue system evaluation using bipartite-play method . authors propose constructing a better automatic evaluation method which is reproducible and low cost . |
| Outcome: | The proposed method correlates strongly with human subjectivity and human evaluation. |
Data Collection and End-to-End Learning for Conversational AI (D19-2)
Copied to clipboard
| Challenge: | tutorial aims to familiarise research community with recent advances in statistical dialogue systems . focus of tutorial is on learning end-to-end from data and their relation to more common modular systems. |
| Approach: | This tutorial aims to familiarise the research community with the latest advances in statistical dialogue systems . the focus of the tutorial is on recently introduced end-to-end learning for dialogue systems and their relation to more common modular systems. |
| Outcome: | This tutorial aims to familiarise the research community with the recent advances in statistical dialogue systems for open-domain and task-based dialogue paradigms. |
Achieving Common Ground in Multi-modal Dialogue (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial focuses on three main topic areas: grounding in human-human communication, dialogue systems and multi-modal interactive systems. |
| Approach: | This tutorial examines the use of computational dialogue research to design grounding modules and behaviors in cutting-edge systems. |
| Outcome: | This tutorial examines the results of recent research on grounding in human-human communication . it shows how these results lead to rich and challenging opportunities for doing grounding more flexible and powerful ways . |
Optimal Summaries for Enabling a Smooth Handover in Chat-Oriented Dialogue (2022.aacl-srw)
Copied to clipboard
| Challenge: | In dialogue systems, it is difficult to provide fully autonomous dialogue . to ensure a good dialogue experience, human operators sometimes need to intervene . |
| Approach: | They conducted large-scale experiments on chat dialogues to determine which type of summary is most useful for handover . abstractive summary plus one utterance immediately before handover and extractive summary consisting of five utterrances immediately before the handover were found to be the most useful . |
| Outcome: | The best summaries were abstractive summary plus one utterance before handover and extractive summary consisting of five utterrances before hand over. |
DiSCoL: Toward Engaging Dialogue Systems through Conversational Line Guided Response Generation (2021.naacl-demos)
Copied to clipboard
| Challenge: | Recent advances in transformer-based language models and their applications to dialogue systems have succeeded to generate fluent and human-like responses. |
| Approach: | They propose to leverage conversational lines as controllable and informative content-planning elements to guide the generation model produce engaging and informative responses. |
| Outcome: | The proposed system generates engaging and informative responses using convlines as controllable and informative content-planning elements. |
Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we discuss the cutting-edge research results and existing challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
| Approach: | This tutorial presents cutting-edge research results and current challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
| Outcome: | This paper reviews the cutting-edge research results and current challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
Deep Learning for Dialogue Systems (C18-3)
Copied to clipboard
| Challenge: | Using deep learning to build robust and scalable spoken dialogue systems is still a challenging task. |
| Approach: | tutorial focuses on an overview of dialogue system development . goal-oriented spoken dialogue systems are most prominent component in virtual personal assistants . |
| Outcome: | This tutorial focuses on an overview of dialogue system development while summarizing the challenges. |
Retrieval-augmented Generation across Heterogeneous Knowledge (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for retrieving knowledge from a single source homogeneous corpus have been gaining increasing attention in the field of natural language processing (NLP) however, they still suffer from the following drawbacks: (i) They are usually trained offline, making the model agnostic to the latest information, e.g., asking a chat-bot about COVID-19. |
| Approach: | They propose to use a single-source homogeneous corpus to generate retrieval-augmented generation models that can learn from the pre-training corpus. |
| Outcome: | The proposed methods have been applied to various knowledge-intensive NLP tasks, but most of the work has focused on retrieving unstructured text documents from Wikipedia. |
Guardrails and Security for LLMs: Safe, Secure and Controllable Steering of LLM Applications (2025.acl-tutorials)
Copied to clipboard
Traian Rebedea, Leon Derczynski, Shaona Ghosh, Makesh Narsimhan Sreedhar, Faeze Brahman, Liwei Jiang, Bo Li, Yulia Tsvetkov, Christopher Parisien, Yejin Choi
| Challenge: | Pretrained generative models provide novel ways for users to interact with computers. |
| Approach: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs along with evaluation methodologies and a detailed security assessment protocol. |
| Outcome: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs, along with evaluation methodologies and a detailed security assessment protocol. |
Neural Retriever and Go Beyond: A Thesis Proposal (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing neural retrievers are developed for pure-text queries, which prevents them from handling multi-modality queries. |
| Approach: | They propose methods to address issues of existing neural retrievers from three angles . they propose new model architectures, IR-oriented pretraining tasks and generating large scale training data . |
| Outcome: | The proposed methods address the abovementioned issues of neural retrievers from three angles and generate large scale training data. |
Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversations (2025.findings-naacl)
Copied to clipboard
Hao Yang, Hongyuan Lu, Xinhua Zeng, Yang Liu, Xiang Zhang, Haoran Yang, Yumeng Zhang, Shan Huang, Yiran Wei, Wai Lam
| Challenge: | a new paradigm for dialogue systems is being developed to mimic human interactions . the current single-step dialogue paradigm lacks the depth and fluidity of human interactions. |
| Approach: | They propose a step-by-step dialogue paradigm that mimics human interactions . they use a dataset to fine-tune existing language models . |
| Outcome: | The proposed system mimics the dynamic nature of human conversations . it is compared with existing paradigms and will be released later this year . |
Polite Chatbot: A Text Style Transfer Application (2023.eacl-srw)
Copied to clipboard
| Challenge: | Creating polite chatbots requires complex setups that require reinforcement learning to produce coherent responses. |
| Approach: | They propose a polite chatbot that can generate coherent responses to given contexts by using a model that transfers neutral sentences into polite ones and trains a dialogue model. |
| Outcome: | The proposed method outperforms baselines in producing polite dialogue responses while staying competitive in terms of coherent to the given context. |
A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing dialogue systems have demonstrated impressive performance conducting fluent and natural-sounding conversations, but they are plagued by the Knowledge Hallucination problem. |
| Approach: | They propose a method that exploits the dialogue-knowledge interaction to reduce hallucination by using external knowledge resources to generate more informative responses. |
| Outcome: | The proposed method reduces hallucination without disrupting other dialogue performance while keeping adaptive to different generation models. |
Exploring Straightforward Methods for Automatic Conversational Red-Teaming (2025.naacl-industry)
Copied to clipboard
George Kour, Naama Zwerdling, Marcel Zalmanovici, Ateret Anaby Tavor, Ora Nova Fandina, Eitan Farchi
| Challenge: | Large language models (LLMs) are increasingly used in business dialogue systems but they also pose security and ethical risks. |
| Approach: | They propose to use off-the-shelf large language models to create red-team attacks by eliciting undesired outputs from an attacker LLM. |
| Outcome: | The proposed models can adapt their attack strategies based on prior attempts, but their effectiveness decreases as the alignment of the target model improves. |
Generate, Evaluate, and Select: A Dialogue System with a Response Evaluator for Diversity-Aware Response Generation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing dialogue systems that generate fluent responses are difficult to evaluate due to the one-to-many nature of dialogue, which means the existence of multiple appropriate responses is not appropriate. |
| Approach: | They propose a generator-evaluator model that evaluates multiple responses generated by a response generator and selects the best response by an evaluators. |
| Outcome: | The proposed model is compared with a baseline system and its outputs were judged to be better than the baseline system. |
MoEL: Mixture of Empathetic Listeners (D19-1)
Copied to clipboard
| Challenge: | Neural network approaches for conversation models have shown to be successful in generating fluent and relevant responses. |
| Approach: | They propose a novel end-to-end approach for modeling empathy in dialogue systems by using Mixture of Empathetic Listeners (MoEL). |
| Outcome: | The proposed model outperforms multitask training baseline in terms of empathy, relevance, and fluency. |
Game-Based Video-Context Dialogue (D18-1)
Copied to clipboard
| Challenge: | Current dialogue systems focus more on textual and speech context knowledge and are usually based on two speakers. |
| Approach: | They propose to use live soccer game videos and Twitch.tv chats to develop visual-grounded dialogue models. |
| Outcome: | The proposed model can generate relevant temporal and spatial event language from live video and chat history while also being relevant to chat history. |
Saying No is An Art: Contextualized Fallback Responses for Unanswerable Dialogue Queries (2021.acl-short)
Copied to clipboard
| Challenge: | despite advances in task-oriented and chit-chat based dialogue systems, many systems rely on static and unnatural responses. |
| Approach: | They propose a neural approach which generates contextually aware responses to user queries . they perform automatic and manual evaluations to demonstrate the efficacy of the system . |
| Outcome: | The proposed approach generates responses which are contextually aware with the user query and say no to the user. |
Knowledge-enhanced Response Generation in Dialogue Systems: Current Advancements and Emerging Horizons (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Knowledge-enhanced Dialogue Systems (KEDS) are a new approach to enhancing human-machine interaction through natural language. |
| Approach: | This tutorial provides an in-depth exploration of Knowledge-enhanced Dialogue Systems (KEDS) it aims to elucidate their significance, highlight advances made using deep learning, and pinpoint the current challenges. |
| Outcome: | The tutorial aims to give attendees a comprehensive understanding of KEDS, and highlight advances made using deep learning and pinpoint the current challenges. |
Transferable Dialogue Systems and User Simulators (2021.acl-long)
Copied to clipboard
| Challenge: | a lack of training data is limiting the development of dialogue systems . we develop a framework for creating dialogue data through self-play between agents . |
| Approach: | They propose a framework that can incorporate new dialogue scenarios through self-play between two agents. |
| Outcome: | The proposed framework is highly effective in bootstrapping the performance of two agents in transfer learning. |
User Interest Modelling in Argumentative Dialogue Systems (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on user interest in dialogue systems depend on explicit user feedback. |
| Approach: | They propose a model to implicitly estimate user interest during argumentative dialogues based on semantically clustered data. |
| Outcome: | The proposed model achieves a classification accuracy of 74.9% and tested with different Artificial Neural Networks (ANN) which new argument would fit the user interest best. |
Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions (2024.eacl-long)
Copied to clipboard
| Challenge: | Using model uncertainty as supervision for deciding when to ask may not be the most effective way to resolve model uncertainty. |
| Approach: | They propose to generate clarification questions based on model uncertainty estimation and compare it to several alternatives to generate questions . |
| Outcome: | The proposed approach improves the model uncertainty of a collaborative dialogue task and shows that it is more effective than other alternatives. |
Multi-Domain Dialogue State Tracking By Neural-Retrieval Augmentation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing approaches for DST are conditioned on previous dialogue states, but the dependency on previous dialogs makes it difficult to prevent error propagation to subsequent turns. |
| Approach: | They propose to create a Neural Index based on dialogue context by analyzing user dialogue and previous turn state and generating a retrieval-guided generation approach. |
| Outcome: | The proposed framework retrieves dialogue context from the index built using unstructured dialogue state and structured user/system utterances. |
A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods of summarizing meetings require complex multi-step pipelines that are intractable. |
| Approach: | They propose an abstractive summary network that adapts to meeting transcripts by hierarchical structure and role vectors. |
| Outcome: | The proposed model outperforms existing methods in both metrics and human evaluation. |
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020.acl-demos)
Copied to clipboard
Qi Zhu, Zheng Zhang, Yan Fang, Xiang Li, Ryuichi Takanobu, Jinchao Li, Baolin Peng, Jianfeng Gao, Xiaoyan Zhu, Minlie Huang
| Challenge: | ConvLab-2 inherits Convlab's framework but integrates more powerful dialogue models and supports more datasets. |
| Approach: | They present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models and perform an end-to-end evaluation. |
| Outcome: | The new tool inherits ConvLab's framework and extends it by integrating many recently proposed state-of-the-art dialogue models. |
DeepPavlov: Open-Source Library for Dialogue Systems (P18-4)
Copied to clipboard
Mikhail Burtsev, Alexander Seliverstov, Rafael Airapetyan, Mikhail Arkhipov, Dilyara Baymurzina, Nickolay Bushkov, Olga Gureenkova, Taras Khakhulin, Yuri Kuratov, Denis Kuznetsov, Alexey Litinsky, Varvara Logacheva, Alexey Lymar, Valentin Malykh, Maxim Petrov, Vadim Polulyakh, Leonid Pugachev, Alexey Sorokin, Maria Vikhreva, Marat Zaynutdinov
| Challenge: | open-source library DeepPavlov is designed for rapid development of dialogue systems. |
| Approach: | open-source library DeepPavlov is tailored for development of conversational agents . the library prioritizes efficiency, modularity and extensibility with the goal to make it easier to develop dialogue systems from scratch . |
| Outcome: | the open-source library DeepPavlov is designed for rapid development of dialogue systems . it supports modular as well as end-to-end approaches to implementation of conversational agents . |
Samvaadhana: A Telugu Dialogue System in Hospital Domain (D19-61)
Copied to clipboard
| Challenge: | a dialogue system for Hospital domain in Telugu is a resource-poor Dravidian language . the system handles various hospital and doctor related queries . |
| Approach: | They propose to model a dialogue system for Hospital domain in Telugu which is a resource-poor Dravidian language. |
| Outcome: | The proposed system achieves a high overall rating and a significantly accurate context-capturing method. |
Continual Few-shot Intent Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing intent detection systems are trained with lots of labeled data over a predefined set of intent classes. |
| Approach: | They propose a prefix-guided lightweight encoder with three auxiliary strategies to prevent catastrophic forgetting and negative knowledge transfer across tasks. |
| Outcome: | The proposed system prevents catastrophic forgetting and encourages positive knowledge transfer across tasks. |
Explicit Use of Topicality in Dialogue Response Generation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing chat dialogue systems only implicitly consider the topic given the context, but not explicitly. |
| Approach: | They propose a dialogue system that responds appropriately following the topic by selecting the entity with the highest “topicality” they define the entity as a noun or compound nouns, and topicality as the degree of speaker awareness directed toward each entity in the dialogue context. |
| Outcome: | The proposed system can follow the topic more than existing systems that only consider the context . |
Schema Encoding for Transferable Dialogue State Tracking (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has focused on deep neural models for task-oriented dialogue systems . however, the neural models require a large dataset for training and a new dataset to be trained on another domain. |
| Approach: | They propose a schema encoder for transferable dialogue state tracking to new domains . they aim to transfer the model to new datasets by encoding new schemas based on the dataset . |
| Outcome: | The proposed method improves the accuracy of the proposed model on multi-domain settings. |
Automating Human Evaluation of Dialogue Systems (2022.naacl-srw)
Copied to clipboard
| Challenge: | a recent study shows that human evaluations of dialogue systems weakly reflect human judgments. |
| Approach: | They propose a BERT-based model that fine-tunes a model with three prediction heads to predict whether the system-generated output is natural, fluent, and informative. |
| Outcome: | The proposed model achieves an average accuracy of 77% over the 3 labels . it also uses three different models to compute the labels compared to three separate models . |
PyOpenDial: A Python-based Domain-Independent Toolkit for Developing Spoken Dialogue Systems with Probabilistic Rules (D19-3)
Copied to clipboard
| Challenge: | a recent development of spoken dialogue systems has enabled deep learning to achieve state-of-the-art performance. |
| Approach: | They propose a Python-based domain-independent, open-source toolkit for spoken dialogue systems. |
| Outcome: | The proposed toolkit extends OpenDial's Java-based architecture and provides new functions for neural dialogue state tracking and action planning. |
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Instruction tuning is emerging in NLP, but has not been explored for dialogue-related tasks. |
| Approach: | They propose an instruction tuning framework for dialogue that leverages natural language instructions with language models to induce zero-shot generalization on unseen tasks. |
| Outcome: | The proposed framework enables good zero-shot performance on unseen datasets and tasks such as dialogue evaluation and intent detection. |
Domain Expert Platform for Goal-Oriented Dialog Collection (2021.eacl-demos)
Copied to clipboard
| Challenge: | a prerequisite for the creation of a goal-oriented neural network dialogue system is a dataset that represents typical dialogue scenarios and includes various semantic annotations. |
| Approach: | They propose a web-based platform for collecting and writing goal-oriented dialogue samples. |
| Outcome: | The proposed platform is language-independent and is currently being used to collect dialogue samples in Latvian . |
GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior methods for detecting out-of-scope (OOS) utterances in text are limited and require a limited amount of data to obtain. |
| Approach: | They propose an orthogonal technique that augments existing data to train better OOS detectors operating in low-data regimes. |
| Outcome: | The proposed method outperforms existing methods on key metrics across three benchmarks and achieves relative gains of 52.4%, 48.9% and 50.3%. |
CRWIZ: A Framework for Crowdsourcing Real-Time Wizard-of-Oz Dialogues (2020.lrec-1)
Copied to clipboard
| Challenge: | Crowdsourcing platforms such as Amazon Mechanical Turk have been effective for collecting large corpora of task-based and open-domain conversational dialogues, but difficulties arise when task- based dialogues require expert domain knowledge or rapid access to domain-relevant information. |
| Approach: | They propose a framework for collecting real-time Wizard of Oz dialogues through crowdsourcing for collaborative, complex tasks. |
| Outcome: | The proposed framework avoids interactions that breach procedures only known to experts while enabling the capture of a wide variety of interactions. |
HiGRU: Hierarchical Gated Recurrent Units for Utterance-Level Emotion Recognition (N19-1)
Copied to clipboard
| Challenge: | Using textual features, our proposed HiGRU models achieve at least 8.7%, 7.5%, 6.0% improvement over the state-of-the-art methods on each dataset. |
| Approach: | They propose a hierarchical gated recurrent unit framework to model word-level inputs and an upper-level GRU to capture contexts of utterance-level embeddings. |
| Outcome: | The proposed framework achieves 8.7%, 7.5%, 6.0% improvement over state-of-the-art methods on three datasets. |
LEGOEval: An Open-Source Toolkit for Dialogue System Evaluation via Crowdsourcing (2021.acl-demo)
Copied to clipboard
| Challenge: | Currently, researchers use automatic metrics and human evaluation to evaluate dialogue systems. |
| Approach: | They propose to use a Python API to easily evaluate dialogue systems using Amazon Mechanical Turk. |
| Outcome: | The open-source toolkit provides a fast, consistent method for reproducing human evaluation results. |
DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing research is limited by general or niche datasets that lack sufficient scale for training dialogue systems. |
| Approach: | They propose a synthetic dialogue generation framework that uses Large Language Models and Chain of Thought reasoning to generate dynamic, domain-specific dialogues with simulated personas and diverse conversational features. |
| Outcome: | The proposed framework outperforms existing frameworks on dialogue summarization and quality increases as the size of the LLM increases from 3B to 8B. |
HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets focused on multiturn dialogue systems focus on text or table information. |
| Approach: | They propose a dataset that consists of crowdsourced conversations grounded on Wikipedia text and tables. |
| Outcome: | The proposed dataset shows that there is still ample opportunity for improvement in the current state of dialogue systems. |
Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | In recent years, neural-network based models have been used for a wide range of tasks, including slot filling and intent classification. |
| Approach: | They propose three neural architectures to model slot filling and intent classification . they propose independent models, joint models and transfer learning models that exploit the mutual benefit of the two tasks simultaneously and scale the model to new domains. |
| Outcome: | The proposed models model SF and IC separately, exploit mutual benefit of the two tasks simultaneously and scale the model to new domains. |
Towards LLM-driven Dialogue State Tracking (2023.emnlp-main)
Copied to clipboard
| Challenge: | emergence of large language models (LLMs) such as GPT3 and ChatGPT has sparked considerable interest in assessing their efficacy across diverse applications. |
| Approach: | They present a framework for a domain-slot instruction tuning method that allows LDST to achieve performance on par with ChatGPT. |
| Outcome: | The proposed framework performs better in zero-shot and few-shot settings than previous SOTA methods. |
Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing research focuses on task-oriented or open-domain dialogue systems with influence skills. |
| Approach: | They propose to define and introduce a category of social influence dialogue systems that influence users’ cognitive and emotional responses. |
| Outcome: | The proposed system is task-oriented or goal-oriented, but it is not open-domain. |
End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2 (2020.acl-main)
Copied to clipboard
| Challenge: | End-to-end dialogue systems with monolithic neural architecture are often trained with input-output utterances without taking into account the entire annotations available in the corpus. |
| Approach: | They propose an end-to-end neural architecture for goal-oriented dialogue systems that addresses both challenges . they propose a modular architecture where modules are optimized individually . |
| Outcome: | The proposed system achieved the top position in the human evaluation task . it is based on a neural architecture that can be integrated with external systems . |
Learning Dialogue Representations from Consecutive Utterances (2022.naacl-main)
Copied to clipboard
| Challenge: | Dialogue Sentence Embedding (DSE) is a self-supervised contrastive learning method that learns effective dialogue representations suitable for a wide range of dialogue-oriented tasks. |
| Approach: | They propose a self-supervised contrastive learning method that learns dialogue representations suitable for a wide range of dialogue tasks. |
| Outcome: | The proposed method outperforms baselines on five dialogue tasks on a few-shot and zero-shot datasets. |
Can ChatGPT’s Performance be Improved on Verb Metaphor Detection Tasks? Bootstrapping and Combining Tacit Knowledge (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for supervised metaphor detection are limited by their performance. |
| Approach: | They propose to use ChatGPT to detect most prevalent verb metaphors among metaphors . they use literal collocations of target verbs and subject-object pairs of verbs to detect them . |
| Outcome: | The proposed method achieves the best performance on the unsupervised verb metaphors detection task compared to existing unsupervised methods or direct prediction using ChatGPT. |
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features (2021.acl-long)
Copied to clipboard
| Challenge: | Existing systems that strive to be informative teachers are difficult to build . knowledge grounded dialogue systems are difficult because of limited training objectives . |
| Approach: | They propose to train a generative neural dialogue model that is controlled to stay faithful to evidence . they propose to use additional inputs to generate more objective responses . |
| Outcome: | The proposed model produces responses that are perceived by humans to be objective and faithful to evidence. |
Maintaining Common Ground in Dynamic Environments (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing task settings focus on creating and maintaining common ground under static contexts and ignore their dynamic aspects. |
| Approach: | They propose a task setting to study the ability of creating and maintaining common ground in dynamic environments. |
| Outcome: | The proposed task setting enables fine-grained evaluation and analysis of various dialogue systems. |
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021.naacl-main)
Copied to clipboard
| Challenge: | Ad hominem attacks target a person's character instead of the position the person is maintaining. |
| Approach: | They propose to use salient n-gram similarity as a soft constraint to reduce the amount of ad hominems generated in Twitter conversations. |
| Outcome: | The proposed method reduces the amount of ad hominems generated in human and dialogue system responses to English Twitter posts by using salient n-gram similarity as a soft constraint. |
The Margarita Dialogue Corpus: A Data Set for Time-Offset Interactions and Unstructured Dialogue Systems (2020.lrec-1)
Copied to clipboard
| Challenge: | Time-Offset Interaction Applications (TOIAs) simulate face-to-face conversations between humans and digital human avatars recorded in the past. |
| Approach: | They propose a methodology for creating the knowledge base for a TOIA, a dialogue corpus, and baselines for single-turn answer retrieval. |
| Outcome: | The proposed method lets the avatar maker list pairs by intuition, guessing what possible questions a user may ask to the avatar. |
Dialogue Act-based Breakdown Detection in Negotiation Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have succeeded in modeling a negotiating agent in natural language that can control both text generation and reasoning in goal-oriented dialogue systems. |
| Approach: | They propose a human-human negotiation dialogue dataset that features increased complexities in terms of the number of possible solutions and a utility function. |
| Outcome: | The proposed method performs comparable to text-based approaches in existing corpora and better results in the proposed dataset. |
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that dialogue systems trained on human conversation data are biased and can produce responses that reflect people’s gender prejudice. |
| Approach: | They propose a novel adversarial learning framework Debiased-Chat to train dialogue models free from gender bias while keeping their performance. |
| Outcome: | The proposed framework significantly reduces gender bias in dialogue models while maintaining the response quality. |
Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems (2020.lrec-1)
Copied to clipboard
| Challenge: | Argumentative dialogue systems and chat bots require a database of arguments that matches their requirements. |
| Approach: | They propose a dialogue system that presents arguments by virtual avatar and synthetic speech to users and allows them to rate the presented content in four different categories. |
| Outcome: | The proposed system evaluates arguments retrieved by two state-of-the-art argument search engines and a system based on traditional web search. |
Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing (P18-2)
Copied to clipboard
| Challenge: | Existing approaches to scalability of dialogue belief tracking are dependent on the ontology of the dialogue . current approaches are not scalable to multi-domain dialogues because of the effort required to define a semantic dictionary for each domain. |
| Approach: | They propose a model that utilizes semantic similarity between dialogue utterances and ontology terms to allow information to be shared across domains. |
| Outcome: | The proposed model outperforms state-of-the-art models in multi-domain dialogue tracking tasks while maintaining high quality. |
Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for meeting summarization rely on transcripts and generate generic summaries, failing to contextualize long discussions and to tailor information to individual preferences and productivity requirements. |
| Approach: | They propose a multi-source approach that considers supplementary materials and generates a summary from this enriched transcript. |
| Outcome: | The proposed model increases summary relevance by 9% and produces more content-rich outputs. |
TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation (2024.acl-long)
Copied to clipboard
| Challenge: | Current methods for Continual Dialogue State Tracking (DST) struggle with catastrophic forgetting and knowledge transfer between tasks. |
| Approach: | They propose a framework for task skill localization and consolidation that enables effective knowledge transfer without relying on memory replay. |
| Outcome: | The proposed framework shows a 7.6% increase in Avg. JGA and 11% rise in BWT metrics over existing state-of-the-art methods. |
Getting To Know You: User Attribute Extraction from Dialogues (2020.lrec-1)
Copied to clipboard
| Challenge: | a new method to extract user attributes from dialogues is needed to improve user understanding. |
| Approach: | They propose to leverage dialogues with conversational agents to automatically extract user attributes from dialogues. |
| Outcome: | The proposed model surpasses retrieval and generation baselines on human evaluation. |
Analysis of Sensation-transfer Dialogues in Motorsports (2024.lrec-main)
Copied to clipboard
| Challenge: | a recent study has examined the effects of subjective ideas on group performance in motorsports. |
| Approach: | They collected dialogues between drivers and engineers in motorsports to test this hypothesis . they defined "sensation" as a unique event unfolding in the mind of a speaker . |
| Outcome: | The results show that the more subjective information interlocutors exchange, the better the group performance in collaborative work. |
FaithDial: A Faithful Benchmark for Information-Seeking Dialogue (2022.tacl-1)
Copied to clipboard
| Challenge: | a new benchmark for hallucination-free dialogues is based on knowledge-based conversational models that generate unsupported utterances . a recent study shows that models that are trustworthy generate unverifiable or factually incorrect statements . |
| Approach: | They propose a data-centric solution to edit hallucinated responses in the Wizard of Wikipedia benchmark. |
| Outcome: | The proposed model improves on the Wizard of Wikipedia benchmark while maintaining engaging conversations. |
Dialogue-AMR: Abstract Meaning Representation for Dialogue (2020.lrec-1)
Copied to clipboard
Claire Bonial, Lucia Donatelli, Mitchell Abrams, Stephanie M. Lukin, Stephen Tratz, Matthew Marge, Ron Artstein, David Traum, Clare Voss
| Challenge: | Abstract Meaning Representation (AMR) does not capture the illocutionary force or speaker’s intended contribution in the broader dialogue context. |
| Approach: | They propose a schema that enriches Abstract Meaning Representation (AMR) it provides a semantic representation for facilitating Natural Language Understanding (NLU) in dialogue systems. |
| Outcome: | The proposed schema provides a semantic representation for facilitating Natural Language Understanding (NLU) in human-robot dialogue systems. |
Benchmarking Large Language Models on Communicative Medical Coaching: A Dataset and a Novel System (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing applications of natural language processing (NLP) focus on patient-centered services, but the potential of NLP to benefit inexperienced doctors remains unexplored. |
| Approach: | They propose a human-AI cooperative framework to assist medical learners in practicing communication skills during patient consultations. |
| Outcome: | The proposed framework enables medical learners to practice communication skills during patient consultations while a coach agent provides immediate, structured feedback. |
GCDST: A Graph-based and Copy-augmented Multi-domain Dialogue State Tracking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to training DST on a single domain ignore information across domains. |
| Approach: | They construct a dialogue state graph to transfer structured features among related domain-slot pairs across domains and encode the graph information of dialogue states by graph convolutional networks. |
| Outcome: | The proposed model improves the performance of the multi-domain DST baseline with the absolute joint accuracy of 2.0% and 1.0% on the MultiWOZ 2.0 and 2.1 dialogue datasets. |
Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for target-guided response generation are inconsistent with human judgement ratings. |
| Approach: | They propose a technique that finds a bridging path between the source and target and uses it to generate transition responses. |
| Outcome: | The proposed technique outperforms baselines on target-guided response generation task. |
Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for unknown intent detection are limited by prior knowledge of class labels. |
| Approach: | They propose to use a Gaussian mixture model to model utterance embeddings with a distribution and inject dynamic class semantic information into Gausssian means. |
| Outcome: | The proposed model performs well on three real task-oriented dialogue datasets in two languages. |
Building Persona Consistent Dialogue Agents with Offline Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve persona consistency are centered around supervised learning or online reinforcement learning (RL). Existing approaches to improve consistency are expensive and require additional training. |
| Approach: | They propose an offline supervised learning framework to improve persona consistency of dialogue systems by punishing and rewarding specific utterances. |
| Outcome: | The proposed framework improves both the persona consistency and dialogue quality of a state-of-the-art social chatbot. |
Constructing Multi-Modal Dialogue Dataset by Replacing Text with Semantically Relevant Images (2021.acl-short)
Copied to clipboard
| Challenge: | Existing training methods for multi-modal dialogue systems rely on image captioning or visual question answering datasets that are irrelevant to the dialogue context. |
| Approach: | They propose to create a 45k multi-modal dialogue dataset with minimal human intervention . they use text dialogue datasets, image-mixed dialogues and contextual-similarity filtering . |
| Outcome: | The proposed dataset can be used as training data for multi-modal dialogue systems . human evaluations show that the model can be effectively used . |
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions (2023.acl-long)
Copied to clipboard
| Challenge: | A moral dialogue system aligned with users’ values could enhance conversation engagement and user connections. |
| Approach: | They propose a framework to train and evaluate moral dialogue systems based on communication mechanisms of morality and a method to construct moral discussions between simulated users and the dialogue system. |
| Outcome: | The proposed framework can train and evaluate moral dialogue systems based on simulated users and their values . |
MuTual: A Dataset for Multi-Turn Dialogue Reasoning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing non-task oriented dialogue systems can yield a relevant and fluent response, but sometimes make logical mistakes because of weak reasoning capabilities. |
| Approach: | They propose a dataset for multi-turn dialogue reasoning that uses annotated dialogues to train a machine to handle various reasoning problems. |
| Outcome: | Empirical results show that state-of-the-art methods only reach 71%, far behind human performance of 94%. |
Towards Building a Spoken Dialogue System for Argument Exploration (2022.lrec-1)
Copied to clipboard
| Challenge: | Argumentative dialogue systems lack a robust natural language understanding framework for complex tasks . drop-down menus hinder the application of natural language learning approaches . |
| Approach: | They propose to integrate a natural language understanding framework into an argumentative dialogue system. |
| Outcome: | The proposed system is compared to a baseline system using a drop-down menu . the drop- down menu convinces, but the willingness to use it is significantly higher . |
A Brief Survey of Textual Dialogue Corpora (2022.lrec-1)
Copied to clipboard
| Challenge: | Several dialogue corpora are available for research purposes, but they do not cover all the necessities of real-world applications. |
| Approach: | They analyze available dialogue corpora and propose possible approaches to create new ones. |
| Outcome: | The proposed corpus of human-human dialogues is based on a list of available dialogue corpora . it covers speakers, size, languages, collection, annotations, and domains . some trends are identified and possible approaches are also discussed . |
MEEP: Is this Engaging? Prompting Large Language Models for Dialogue Evaluation in Multilingual Settings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing metrics for engagingness evaluate the response without the conversation history, are designed for one dataset, or have limited correlation with human annotations. |
| Approach: | They propose to use large language models to evaluate engagingness in dialogue . they propose to include prompts and translated prompts in the model . |
| Outcome: | The proposed model outperforms existing methods on evaluation of engagingness in dialogue across languages. |
ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) inputs are limited and lack data. |
| Approach: | They propose a new architecture that extends Prototypical Networks to process in-domain and OOD sentences via Mutual Information Maximization objective. |
| Outcome: | The proposed method significantly improves performance up to 20% for OOD detection in low resource settings of text classification. |
TED-Q: TED Talks and the Questions they Evoke (2020.lrec-1)
Copied to clipboard
| Challenge: | Evoked questions represent a hitherto unexplored type of linguistic data, promising to open up important new lines of research. |
| Approach: | They propose a method to annotate TED-talks with the questions they evoke and, where available, the answers to these questions. |
| Outcome: | The proposed method is designed to scale up, relying on crowdsourcing by non-expert annotators, with its utility for Natural Language Processing in mind. |
Advancing E-commerce Merchants Telemarketing with Synthetic Data-Driven LLMs (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are proving broadly applicable across diverse industries, including e-commerce. |
| Approach: | They propose a hybrid data synthesis framework that unifies the input schema with profile and strategy designed by top sales and extracts them via a Multi-task paradigm. |
| Outcome: | The proposed model reaches the performance level of the top 25% of human sales in terms of the final marketing results. |
Personalized Response Generation via Generative Split Memory Network (2021.naacl-main)
Copied to clipboard
| Challenge: | Despite the success of text generation and dialogue systems, how to endow a text generation system with personality traits remains under-investigated. |
| Approach: | They propose a model to generate personalized responses on reddit using user profiles and posting histories. |
| Outcome: | The proposed model improves over the state-of-the-art response generation models. |
CausalScore: An Automatic Reference-Free Metric for Assessing Response Relevance in Open-Domain Dialogue Systems (2025.coling-main)
Copied to clipboard
| Challenge: | Existing metrics for dialogue quality evaluation show low correlation with human judgements . current metrics do not accurately evaluate dialogue responses based on dialogue history . |
| Approach: | They propose a new metric measuring causal strength between dialogue histories and responses . they collect a dialogue dataset with human-annotated causal relations and pairwise human judgements . |
| Outcome: | The proposed metric outperforms existing state-of-the-art metrics in human judgements . it is based on a dialogue dataset with human-annotated causal relations and human judgement sets . |
Leveraging Explicit Procedural Instructions for Data-Efficient Action Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | a paper presents a data-efficient solution to constructing task-oriented dialogue systems . large language models have shown success in modeling such dialogues, but they require large quantities of data . |
| Approach: | They propose a system that leverages explicit instructions from agent guidelines . they propose dialogue-document matching and action-oriented masked language modeling . |
| Outcome: | The proposed system improves accuracy predicting in- and out-of-distribution actions while preserving high performance in settings with low or sparse data. |
Learning to Predict Persona Information for Dialogue Personalization without Explicit Persona Description (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to personalize dialogue agents rely on explicit persona descriptions during inference, which severely limits their application in real-world scenarios. |
| Approach: | They propose a method that learns to predict persona information based on the dialogue history to personalize dialogue agents without relying on explicit persona descriptions during inference. |
| Outcome: | The proposed method improves the consistency and engagingness of generated responses when conditioning on the predicted profile of the dialogue agent. |
Assessing Dialogue Systems with Distribution Distances (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation metrics focus on turnlevel quality, which is not well suited for open-end dialogue tasks. |
| Approach: | They propose to measure the performance of a dialogue system by computing the distributionwise distance between its generated conversations and real-world conversations. |
| Outcome: | The proposed metrics correlate better with human judgments than existing metrics on dialogue systems. |
EventWeave: A Dynamic Framework for Capturing Core and Supporting Events in Dialogue Systems (2026.acl-long)
Copied to clipboard
Zhengyi Zhao, Shubo Zhang, Yiming Du, Bin Liang, Baojun Wang, Zhongyang Li, Binyang Li, Kam-Fai Wong
| Challenge: | Existing dialogue systems process conversational turns in isolation, overlooking event structures that guide natural interactions. |
| Approach: | They propose a framework that explicitly models relationships between conversational events to generate more contextually appropriate dialogue responses. |
| Outcome: | Experiments on three dialogue datasets show that the proposed approach produces more natural responses while requiring less computational overhead. |
Learning the Human Judgment for the Automatic Evaluation of Chatbot (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing evaluation methods for dialogue systems rely on human judges to label quality of generated text. |
| Approach: | They propose a machine learning approach to reduce the effort of human evaluation by learning the human judgment on comparing two generative dialogue systems. |
| Outcome: | The proposed method reduces the effort of human evaluation by learning which generative models is better in each dialog context. |
Who Is Speaking to Whom? Learning to Identify Utterance Addressee in Multi-Party Conversations (D19-1)
Copied to clipboard
| Challenge: | In multi-party conversations, addressee information is not always explicit . researchers have spent great efforts to understand conversations between two participants, which is known as multi-part conversation. |
| Approach: | They propose a who-to-whom model which models users and utterances in a conversation session jointly in an interactive way. |
| Outcome: | The proposed model outperforms baseline models on the Ubuntu Multi-Party Conversation Corpus and shows consistent improvements. |
Training a Broad-Coverage German Sentiment Classification Model for Dialog Systems (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing sentiment data sets are not available for sentiment analysis. |
| Approach: | They propose to combine a German sentiment corpus with existing resources to train a general-purpose German sentiment classification model. |
| Outcome: | The proposed model trains a general-purpose German sentiment classification model . the data set contains 5.4 million labelled samples . |
Beyond Linguistic Cues: Fine-grained Conversational Emotion Recognition via Belief-Desire Modelling (2024.lrec-main)
Copied to clipboard
| Challenge: | Emotion recognition in conversation (ERC) is essential for dialogue systems to identify the emotions expressed by speakers. |
| Approach: | They propose a method that incorporates both belief and desire to accurately identify emotions by extracting emotion-eliciting events from utterances and construct graphs that represent beliefs and desires in conversations. |
| Outcome: | The proposed model outperforms existing models on four popular ERC datasets and validates its performance with multiple state-of-the-art models. |
Evaluating Sampling-based Filler Insertion with Spontaneous TTS (2022.lrec-1)
Copied to clipboard
| Challenge: | Injecting fillers into spoken dialogue systems has a rich history of study . ambiguity of filler occurrence and inter-speaker difference make modeling and evaluation difficult. |
| Approach: | They propose an objective score for filler insertion using sampling-based sampling . they build three models trained on two single-speaker spontaneous corpora and evaluate them with FPP and perceptual tests. |
| Outcome: | The proposed model is useful in analysis but does not correlate well with perceptual MOS. |
MultiDM-GCN: Aspect-guided Response Generation in Multi-domain Multi-modal Dialogue System using Graph Convolutional Network (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research suggests that engaging conversations include visual cues (e.g., a video or images) or audio cue. |
| Approach: | They propose a multi-modal conversational framework that generates the responses following the different aspects of a product or service to cater to the user's needs. |
| Outcome: | The proposed framework outperforms baselines for the task-oriented dialogue setup. |
Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system. |
| Approach: | They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards. |
| Outcome: | Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics. |
Multi-User MultiWOZ: Task-Oriented Dialogues among Multiple Users (2023.findings-emnlp)
Copied to clipboard
Yohan Jo, Xinyan Zhao, Arijit Biswas, Nikoletta Basiou, Vincent Auvray, Nikolaos Malandrakis, Angeliki Metallinou, Alexandros Potamianos
| Challenge: | a dataset of task-oriented dialogues assume conversations between the agent and one user at a time . but multi-user task-orientated dialogues are richer, containing deliberation and deliberations . a novel task is proposed to rewrite a task-focused query that retains only task-relevant information . |
| Approach: | They propose to rewrite a task-oriented chat between two users as a concise task-orientated query that retains only task-relevant information and is directly consumable by the dialogue system. |
| Outcome: | The proposed method surpasses existing models on multi-user dialogues and generalizes to unseen domains. |
Grounding Conversations with Improvised Dialogues (2020.acl-main)
Copied to clipboard
| Challenge: | Effective dialogue involves grounding, the process of establishing mutual knowledge that is essential for communication between people. |
| Approach: | a new study uses a corpus of yes-and-turns to analyze improv dialogues . they find that dialogue is a collaborative process by which partners coordinate via turns or acts to jointly construct a common world state. |
| Outcome: | a new study fine-tunes chit-chat dialogue systems with their corpus to encourage more grounded, relevant conversation. |
Learning Improvised Chatbots from Adversarial Modifications of Natural Language Feedback (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, user feedback contains extraneous sequences hindering their usefulness as a training sample. |
| Approach: | They propose a generative adversarial model that converts noisy feedback into a plausible natural response in a conversation and fools the discriminator which distinguishes feedback from natural responses. |
| Outcome: | The proposed model improves the original chatbot performance from 69.94%to 75.96% in ranking correct responses on the PERSONACHATdataset. |
Injecting Salesperson’s Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent research in dialogue systems focuses on task-oriented (TOD) and open-domain (chit-chat) dialogues. |
| Approach: | They propose to use chit-chat to simulate task-oriented dialogues to train sales agents. |
| Outcome: | The proposed model improves coherence and reduces aggression, improving model learning for sales-customer interactions. |
Continual Dialogue State Tracking via Example-Guided Question Answering (2023.emnlp-main)
Copied to clipboard
Hyundong Cho, Andrea Madotto, Zhaojiang Lin, Khyathi Chandu, Satwik Kottur, Jing Xu, Jonathan May, Chinnadhurai Sankar
| Challenge: | Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services causes catastrophic forgetting. |
| Approach: | They propose to reformulate dialogue state tracking (DST) as a bundle of example-guided question answering tasks to minimize the task shift between services. |
| Outcome: | The proposed model achieves state-of-the-art performance on DST continual learning metrics without relying on any complex regularization or parameter expansion methods. |
Modeling with Recurrent Neural Networks for Open Vocabulary Slots (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to filling slots that take on values from a virtually unlimited set have been lacking in the natural language area. |
| Approach: | They propose a new attention-based recurrent neural network (RNN) model that captures the concept: Understanding the role of a word may vary according to how long a reader focuses on a particular part of . sentence. |
| Outcome: | The proposed model outperforms existing models with respect to discovering ‘open-vocabulary’ slots without any external information, such as a named entity database or knowledge base. |
ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems (2023.acl-long)
Copied to clipboard
| Challenge: | evaluating commonsense in dialogue systems remains an open challenge . despite the success of open-domain dialogue systems, systems struggle to produce commonsensical responses as humans do. |
| Approach: | They propose an event commonsense evaluation metric empowered by commonsensence knowledge bases. |
| Outcome: | The proposed metric achieves higher correlations with human judgments than baselines. |
SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Empirical studies show that supervised learning is extremely effective in in-domain datasets and models trained on SuperDialseg can achieve good generalization ability on out-of-domain data. |
| Approach: | They propose a supervised definition of dialogue segmentation points using document-grounded dialogues and a large-scale supervised dataset called SuperDialseg. |
| Outcome: | The proposed model can achieve good generalization ability on out-of-domain data. |
Classifying Sluice Occurrences in Dialogue (L18-1)
Copied to clipboard
| Challenge: | Ellipsis is an important challenge for natural language processing systems, says a new paper . previous work on ellipsis focused on news data, but sluicing presents a challenge for dialogue systems . |
| Approach: | They describe a corpus of 4100 sluice occurrences from the NYTimes Gigaword corpus . they build a classifier model to automatically classify slujce . |
| Outcome: | The proposed corpus contains 4100 sluice occurrences, with an accuracy of 67% . the work will support empirical research into slujcing in dialogue systems . |
Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers (P18-1)
Copied to clipboard
| Challenge: | a novel task of predicting adverbial presupposition triggers is useful for natural language generation . a focus is on a new attention mechanism for predicting presuposition trigger . |
| Approach: | They propose a new attention mechanism for predicting adverbial presupposition triggers . they propose to augment a baseline neural network without additional trainable parameters . |
| Outcome: | The proposed model outperforms baseline models in predicting adverbial presupposition triggers. |
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems (2022.acl-long)
Copied to clipboard
| Challenge: | Moral integrity corpus captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs). |
| Approach: | They propose a resource that captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs). |
| Outcome: | The proposed resource captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs). |
Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Imlicit user feedback, user emotions and demographic information are promising sources for improving the accuracy and user engagement of dialogue responses, but the impact of such information on task completion and factual consistency is not known. |
| Approach: | They introduce the first English task-oriented and document-grounded dialogue dataset annotated with this information. |
| Outcome: | The proposed dataset shows that the model's responses are more informative and factual consistent. |
CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues (2021.naacl-main)
Copied to clipboard
Bo-Hsiang Tseng, Shruti Bhargava, Jiarui Lu, Joel Ruben Antony Moniz, Dhivya Piraviperumal, Lin Li, Hong Yu
| Challenge: | Traditionally, anaphora resolution and ellipses resolution are limited in dialogues . despite rapid progress in dialogue systems, several difficulties remain . |
| Approach: | They propose a joint learning framework for modeling coreference resolution and query rewriting for complex, multi-turn dialogues. |
| Outcome: | The proposed model outperforms the state-of-the-art model on a rewritten dialogue dataset. |
Knowledge-Driven Slot Constraints for Goal-Oriented Dialogue Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Traditional goal-oriented dialogue systems allow execution of validation rules as a post-processing step after slots have been filled which can lead to error accumulation. |
| Approach: | They propose a task of constraint violation detection based on knowledge-driven slot constraints . they propose methods to integrate external knowledge into the system and compare it to traditional rule-based pipeline approach . |
| Outcome: | The proposed task compares to the existing system and a rule-based pipeline. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
There Are a Thousand Hamlets in a Thousand People’s Eyes: Enhancing Knowledge-grounded Dialogue with Personal Memory (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for knowledge selection focus on relevance between knowledge and dialogue context, ignoring personal preference for knowledge. |
| Approach: | They propose to introduce personal memory into knowledge selection in chatbots to address personalization issue by integrating personal memory and inverse mapping into a closed loop. |
| Outcome: | The proposed method outperforms existing methods significantly on automatic evaluation and human evaluation. |
Hello Again! LLM-powered Personalized Agent for Long-term Dialogue (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue systems focus on brief single-session interactions, neglecting real-world needs for long-term companionship and personalized interactions. |
| Approach: | They propose a model-agnostic framework for long-term dialogue agents . they use event summary and persona management to enable reasoning . |
| Outcome: | The proposed framework incorporates three independently tunable modules dedicated to event perception, persona extraction, and response generation. |
AttnIO: Knowledge Graph Exploration with In-and-Out Attention Flow for Knowledge-Grounded Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for retrieving proper knowledge relevant to conversational context use only KG structure . empirical evaluations present a marked performance improvement of AttnIO compared to all baselines in OpenDialKG dataset . |
| Approach: | They propose a dialog-conditioned path traversal model that makes full use of rich structural information in KG . they show a marked performance improvement compared to baselines in OpenDialKG a KG dataset . |
| Outcome: | The proposed model makes full use of rich structural information in KG structure . it can be trained to generate an adequate knowledge path even when paths are not available . |
Information-Theoretic Text Hallucination Reduction for Video-grounded Dialogue (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing video-grounded dialogue systems suffer from text hallucination problem due to learning spurious correlations from the fact that answer sentences in the dataset usually include the words of input texts. |
| Approach: | They propose to decode an answer sentence to a question using video and dialogue contexts. |
| Outcome: | The proposed framework shows that it generates adequate conversational responses to the queries of humans while following up on video and dialogue context. |
Analysing the role of lexical and temporal information in turn-taking through predictability (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of spoken dialogue systems do not address which information sources drive predictions. |
| Approach: | They examine the role of lexical-temporal features on the predictability of turn structure by examining PairwiseTurnGPT, a full-duplex model of spoken dialogue transcripts. |
| Outcome: | The proposed model can produce fluent conversational output, but it does not guarantee realistic turn-taking behaviour. |
Mirages. On Anthropomorphism in Dialogue Systems (2023.emnlp-main)
Copied to clipboard
| Challenge: | Automated dialogue systems are anthropomorphised by developers and personified by users. |
| Approach: | They propose to examine linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise thereof. |
| Outcome: | The proposed systems are anthropomorphised and personified by users . linguistic factors can also be used to reinforce gender stereotypes and conceptions of acceptable language. |
Learning Geometry-Aware Representations for New Intent Discovery (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for intent classification fail to distinguish new intents due to intertwined centers . a novel framework that learns geometry-aware representations to maximally separate all intents is proposed . |
| Approach: | They propose a new intent discovery framework that learns geometry-aware representations to maximally separate all intents. |
| Outcome: | The proposed framework achieves a new state-of-the-art performance on three benchmarking datasets. |
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2022.findings-acl)
Copied to clipboard
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang
| Challenge: | Dialogue safety problems severely limit the real-world deployment of generative conversational models. |
| Approach: | They propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings. |
| Outcome: | The proposed taxonomy captures unsafe behaviors in human-bot dialogue settings with rich context-sensitive unsafe examples. |
Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being utilized as AI negotiation agents . however, prior research on LLMs lacks a systematic evaluation of their diverse capabilities in negotiation. |
| Approach: | They propose to analyze the multifaceted capabilities of Large Language Models (LLMs) across diverse dialogue scenarios throughout the stages of a typical negotiation interaction. |
| Outcome: | The proposed model outperforms GPT-4 in many negotiation tasks while identifying specific challenges, such as making subjective assessments and generating contextually appropriate, strategically advantageous responses. |
Collecting and Analyzing Dialogues in a Tagline Co-Writing Task (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, most studies on dialogue systems focus on problemsolving dialogues and relatively little research has been done on systems that can engage in creative collaboration with users. |
| Approach: | They designed a tagline co-writing task in which two people collaborate to create taglines via text chat and collected dialogue logs, editing logs and questionnaire results. |
| Outcome: | The proposed task involved a tagline co-writing task in which two people collaborate to create taglines via text chat, and collected dialogue logs, editing logs and questionnaire results. |
Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot dialogue state tracking (DST) involve embedding prompts into language models, but these methods have inherent limitations. |
| Approach: | They propose a plug-and-play architecture designed for zero-shot dialogue state tracking (DST) dual low-rank adaptation targets dialogue context processing and prompt optimization without incurring additional inference latency. |
| Outcome: | The proposed architecture outperforms baseline methods on multi-domain datasets and the MultiWOZ dataset. |
A Comprehensive Evaluation of Incremental Speech Recognition and Diarization for Conversational AI (2020.coling-main)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems are increasingly powerful and more numerous with several options existing as a service. |
| Approach: | They evaluate the most popular automatic speech recognition systems with metrics and experiments designed with these standards in mind. |
| Outcome: | The most popular ASR systems are Microsoft and IBM, and none are suitable for natural spontaneous conversations in real-time. |
Combining Discourse Coherence with Large Language Models for More Inclusive, Equitable, and Robust Task-Oriented Dialogue (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are capable of generating well-formed responses, but they struggle in goal-oriented settings. |
| Approach: | They propose a discourse-aware multimodal task-oriented dialogue system that combines discourse theories with offline LLM generation. |
| Outcome: | The proposed system reduces misunderstandings in the dialect of African-American Vernacular English from 93% to 57%. |
EmoCharacter: Evaluating the Emotional Fidelity of Role-Playing Agents in Dialogues (2025.naacl-long)
Copied to clipboard
| Challenge: | EmoCharacter evaluates emotional fidelity of role-playing agents in dialogues . current evaluations focus on personality fidelity, tone imitation, and knowledge consistency . |
| Approach: | They propose a benchmark to assess emotional fidelity of role-playing agents in dialogues using large language models. |
| Outcome: | The proposed benchmark measures emotional fidelity of role-playing agents and the characters they portray. |
Controllable Dialogue Simulation with In-context Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to generate annotated dialogues require crowdsourcing, which is expensive and time-consuming. |
| Approach: | They propose a dialogue simulation method based on large language model in-context learning that generates new dialogues and annotations in a controllable way. |
| Outcome: | The proposed method can expand a small set of dialogue data with minimum or zero human involvement and parameter update. |
LSTDial: Enhancing Dialogue Generation via Long- and Short-Term Measurement Feedback (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue systems do not utilize quality dimensions specifically designed for dialogue evaluation to guide the response generation during training. |
| Approach: | They propose a two-stage framework which generates and utilizes conversation evaluation as explicit feedback during training. |
| Outcome: | The proposed framework generates and utilizes conversation evaluation as explicit feedback during training. |
Measuring the Robustness of Reference-Free Dialogue Evaluation Systems (2025.coling-main)
Copied to clipboard
| Challenge: | Advancements in dialogue systems powered by large language models have outpaced the development of reliable evaluation metrics. |
| Approach: | They propose a benchmark to evaluate the robustness of reference-free dialogue metrics against four categories of adversarial attacks. |
| Outcome: | The proposed benchmarks show that the two axes of reliability are not always aligned . the findings motivate the development of nuanced evaluation frameworks to address real-world dialogue challenges. |
PsyProbe: Proactive and Interpretable Dialogue through User State Modeling for Exploratory Counseling (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to mental health dialogue are reactive and lack systematic user state modeling for proactive therapeutic exploration. |
| Approach: | They propose a dialogue system designed for the exploration phase of counseling that systematically tracks user psychological states through the PPPPPI framework augmented with cognitive error detection. |
| Outcome: | The proposed system outperforms baseline and ablation modes in automatic evaluation and expert evaluation by a certified counselor. |
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks. |
| Approach: | They propose methods for automatically creating adversarial negative training data . they use mask-and-fill and keyword-guided approaches to generate negative examples . |
| Outcome: | The proposed approaches outperform baseline models in providing informative negative examples for training dialogue systems. |
DialogBench: Evaluating LLMs as Human-like Dialogue Systems (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate LLMs' abilities for task completion as assistant AI. |
| Approach: | They propose a dialogue evaluation benchmark that contains 12 dialogue tasks to evaluate LLMs' capabilities as human-like dialogue systems. |
| Outcome: | The proposed benchmark contains 12 tasks to evaluate LLMs' capabilities . it shows that instruction tuning improves human likeness, but not as human-like systems . |
Semantic Representation for Dialogue Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for dialogue modeling lack ability to represent core semantics, such as ignoring important entities. |
| Approach: | They develop an algorithm to construct dialogue-level AMR graphs from sentence-level data and explore two ways to incorporate AMRs into dialogue modeling. |
| Outcome: | The proposed model is superior to existing models on dialogue understanding and response generation tasks. |
Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to estimate turn and dialogue level user satisfaction employ hand-crafted features and rely on complex annotation schemes, which reduce generalizability of the trained models. |
| Approach: | They propose to use an adaptive multi-task loss function to minimize hand-crafted features to estimate user satisfaction at turn level from an end user perspective. |
| Outcome: | The proposed model improves on 28 Alexa domains, two dialogue systems and three user groups on a set of user-generated dialogues from 28 Alexia domain and 28 Alexis domains. |
Supervised Prototypical Contrastive Learning for Emotion Recognition in Conversation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to capture emotions in conversation (ERC) lack the correlation between emotions and semantics, resulting in many challenges. |
| Approach: | They propose a Supervised Prototypical Contrastive Learning (SPCL) loss for the ERC task . they use a Prototype Network to leverage the supervised contrastive learning approach . |
| Outcome: | The proposed approach outperforms CoG-BART's proposed approach on three widely used benchmarks and shows that it is effective on multiple scenarios. |
KNN-Contrastive Learning for Out-of-Domain Intent Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for OOD intent classification are limited to regions with compact or simply-connected features, which assumes no OOD intentions reside. |
| Approach: | They propose a method that uses k-nearest neighbors to learn discriminative semantic features that are more conducive to OOD detection. |
| Outcome: | The proposed method improves OOD detection performance while requiring no restrictions on feature distribution. |
Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for backchannel prediction relied on turn-based or artificially balanced datasets. |
| Approach: | They propose a method for real-time, continuous backchannel prediction using a fine-tuned Voice Activity Projection model. |
| Outcome: | The proposed method outperforms baseline methods in timing and type prediction tasks in real-world environments. |
Suggest me a movie for tonight: Leveraging Knowledge Graphs for Conversational Recommendation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show that knowledge graphs are incomplete since they do not contain all factual information present on the web. |
| Approach: | They propose to use knowledge graphs to improve the performance of conversational recommender systems by incorporating pre-trained embeddings from subgraphs and positional embeddments into their models. |
| Outcome: | The proposed method improves by 5.62% over the state-of-the-art method on multiple metrics on the recommendation task. |
Self-Supervised Dialogue Learning (P19-1)
Copied to clipboard
| Challenge: | Existing dialogue systems have failed to capture the order of utterances in coherent dialogues. |
| Approach: | They propose a self-supervised learning task to capture the flow of dialogues . they propose 'inconsistent order detection' task to predict whether utterance is ordered or misordered . |
| Outcome: | The proposed methods can be applied to open-domain and task-oriented dialogue scenarios and achieve state-of-the-art performance on the OpenSubtitiles and Movie-Ticket Booking datasets. |
CTSM: Combining Trait and State Emotions for Empathetic Response Model (2024.lrec-main)
Copied to clipboard
| Challenge: | Empathetic response generation attempts to empower dialogue systems to perceive speakers’ emotions and generate empathetic responses accordingly. |
| Approach: | They propose to combine trait and state emotions for Empathetic Response Model to enable dialogue systems to perceive speakers' emotions and generate empathetic responses accordingly. |
| Outcome: | The proposed model outperforms state-of-the-art models and generates more empathetic responses. |
Topic Spotting using Hierarchical Networks with Self Attention (N19-1)
Copied to clipboard
| Challenge: | Existing systems struggle to have consistent long term conversations with the users and fail to build rapport. |
| Approach: | They propose a hierarchical model with self attention for topic spotting . they compare it to previous proposed techniques for topic detection . |
| Outcome: | The proposed model outperforms existing models for topic spotting and deep models for text classification in an online setting. |
Evaluating Coherence in Dialogue Systems using Entailment (N19-1)
Copied to clipboard
| Challenge: | Evaluating open-domain dialogue systems is difficult due to the diversity of possible correct answers. |
| Approach: | They propose a set of metrics for evaluating topic coherence using distributed sentence representations and calculable approximations of human judgment using conversational coherency. |
| Outcome: | The proposed metrics can be used as a surrogate for human judgment based on conversational coherence on large-scale datasets and provide an unbiased estimate for the quality of the responses. |
Does Gender Matter? Towards Fairness in Dialogue Systems (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies have shown that AI is unfair in many real-world applications such as computer vision and recommendations. |
| Approach: | They propose to use a benchmark dataset to study the fairness of dialogue systems to understand their bias. |
| Outcome: | The proposed methods reduce the bias in dialogue systems significantly. |
Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue (2020.coling-main)
Copied to clipboard
| Challenge: | In many domains, dialogue systems need to work collaboratively with users to reconstruct meaning . this requires a system that can give targeted, effective feedback about the system’s understanding . |
| Approach: | They propose a system that collaborates on reference tasks that distinguish arbitrarily varying color patches from similar distractors and use crowd workers to test their approach. |
| Outcome: | The proposed system can distinguish varying color patches from distractors and elicit correct answers that the system understands. |
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Current-day large language models generate coherent, grammatical, and seemingly meaningful text, but are prone to hallucinating incorrect information. |
| Approach: | They propose to ‘subtract’ parameters of a model trained to hallucinate from a dialogue response generation model to ‘negate’ the contribution of such hallucinatedexamples from it. |
| Outcome: | The proposed method reduces hallucinations and discourages extractive responses, which are often a consequence of reducing hallucines by encouraging copy-pasting of document spans. |
CoPrUS: Consistency Preserving Utterance Synthesis towards more realistic benchmark dialogues (2025.coling-main)
Copied to clipboard
| Challenge: | Large-scale Wizard-Of-Oz dialogue datasets lack certain types of utterances, which would make them more realistic. |
| Approach: | They propose to use a large language model to create and repair communication errors in an automatic pipeline. |
| Outcome: | The proposed method is based on linguistic theory and uses a state-of-the-art Large Language Model (LLM) to create the error and repair it. |
HEISIR: Hierarchical Expansion of Inverted Semantic Indexing for Training-free Retrieval of Conversational Data using LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for retrieving dialogue data often face challenges in capturing semantic intent or require extensive labeling and fine-tuning. |
| Approach: | They propose a framework that enhances semantic understanding in conversational data retrieval by optimizing data ingestion. |
| Outcome: | The proposed framework outperforms fine-tuned models in retrieval and intent analysis. |
Correction of Errors in Preference Ratings from Automated Metrics for Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods are over-confident in assigning significant differences between systems . Currently, the most reliable evaluation methods for text generation are human-based evaluations. |
| Approach: | They propose to combine human ratings with automated ratings to reduce the amount of human ratings needed to arrive at robust results. |
| Outcome: | The proposed evaluation protocol reduces the amount of human ratings by 50% while yielding the same evaluation outcome as the pure human evaluation in 95% of cases. |
Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely used in NLP, but their social or cultural impact is under-explored. |
| Approach: | They build a dataset consisting of numerous **C**hinese **C*omical **C***rosstalk scripts, which is for a popular Chinese performing art called ‘Xiangsheng’ or ‘’ since 1800s. |
| Outcome: | The proposed approach can generate humor as humans do, but it is still in its infancy. |
AirDialogue: An Environment for Goal-Oriented Dialogue Research (D18-1)
Copied to clipboard
| Challenge: | Recent advances in dialogue generation have inspired a number of studies on dialogue systems . however, current datasets are limited in size and the environment for training agents is relatively unsophisticated. |
| Approach: | They propose to use a context-generator to generate travel and flight restrictions to train agents. |
| Outcome: | The proposed model achieves a score of 0.17 while humans can reach 0.91 . the proposed model is based on a large dataset that contains 301,427 goal-oriented conversations . |
TK-KNN: A Balanced Distance-Based Pseudo Labeling Approach for Semi-Supervised Intent Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised methods for detecting intent generate a large amount of unlabeled data . labeling data requires substantial human effort, and picking an imbalanced set of examples could lead to poor labels. |
| Approach: | They propose a balanced distance-based pseudo-labeling approach for semisupervised intent classification . they use a ranking-based approach to select samples with a model prediction confidence . |
| Outcome: | The proposed method outperforms existing models on popular datasets. |
Analysis of Dialogue in Human-Human Collaboration in Minecraft (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have focused on developing dialogue systems that enable collaborative work, but few studies have centered on creative collaborative work. |
| Approach: | They collected 500 dialogues of human-human collaboration in Minecraft as a basis for developing a dialogue system that enables creative collaborative work. |
| Outcome: | The proposed system can be used to create a collaborative garden in Minecraft and collect text chats, action logs, and subjective evaluations. |
Data Collection for Empirically Determining the Necessary Information for Smooth Handover in Dialogue (2022.lrec-1)
Copied to clipboard
| Challenge: | Despite advances in deep learning, dialogue systems struggle to achieve fully autonomous transactions with users. |
| Approach: | They conducted an experiment in which two operators switched periodically while performing chat, consultation, and sales tasks in dialogue. |
| Outcome: | The results show that adjacency pairs are useful for recording conversation history . key-value pairs are also useful when there are underlying tasks, such as consultation and sales . |
Dialogue Systems Can Generate Appropriate Responses without the Use of Question Marks?– a Study of the Effects of “?” for Spoken Dialogue Systems – (2024.lrec-main)
Copied to clipboard
Tomoya Mizumoto, Takato Yamazaki, Katsumasa Yoshikawa, Masaya Ohagi, Toshiki Kawamoto, Toshinori Sato
| Challenge: | Existing systems for spoken dialogues do not append question marks to recognized queries . lack of punctuation marks in spoken dialogue can negatively impact comprehension . |
| Approach: | They investigate the impact of question marks on spoken dialogue systems . they analyze examples to determine which types of utterances have the impact . |
| Outcome: | The proposed method shows that question marks have a significant impact on spoken dialogue systems. |
DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue (2021.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks do not have enough annotations to analyze video-grounded dialogue systems and understand their capabilities and limitations in isolation. |
| Approach: | They present a Diagnostic Dataset for Video-grounded dialogue with minimal biases and detailed annotations for the different types of reasoning over the spatio-temporal space of video. |
| Outcome: | The proposed system is based on 11k CATER synthetic videos and contains 10 instances of 10-round dialogues for each video. |
RtGender: A Corpus for Studying Differential Responses to Gender (L18-1)
Copied to clipboard
| Challenge: | Prior work on linguistic gender difference and communications about gender has focused on language about or portraying persons of a particular gender. |
| Approach: | They present a multi-genre corpus of 25M comments from five socially and topically diverse sources tagged for the gender of the addressee and 30k annotations for sentiment and relevance of these responses. |
| Outcome: | The proposed dataset shows that responses to women are more emotive and about the speaker as an individual (rather than about the content being responded to). |
I run as fast as a rabbit, can you? A Multilingual Simile Dialogues Datasets (2023.findings-acl)
Copied to clipboard
| Challenge: | A simile is a figure of speech that compares two different things via shared properties. |
| Approach: | They propose a multilingual simile dialogue dataset that can be used to study similes in real-life scenarios. |
| Outcome: | The proposed dataset is the largest manually annotated simile dataset and contains both English and Chinese data. |
Semi-supervised New Slot Discovery with Incremental Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to discover new slots rely on unsupervised slot induction or domain adaptation, and are limited in transferring prior knowledge to new slots. |
| Approach: | They propose a Semi-supervised Incremental Clustering method to discover new slots with existing linguistic annotation models and limited known slot data. |
| Outcome: | The proposed method significantly outperforms state-of-the-art models on five public datasets. |
Task-Oriented Conversation Generation Using Heterogeneous Memory Networks (D19-1)
Copied to clipboard
| Challenge: | Existing memory networks do not perform well when leveraging heterogeneous information from different sources. |
| Approach: | They propose to use user utterances, dialogue history and background knowledge tuples to integrate external knowledge into a neural dialogue model. |
| Outcome: | The proposed model outperforms the state-of-the-art data-driven task-oriented dialogue models on real-world datasets. |
Predicting Nods by using Dialogue Acts in Dialogue (L18-1)
Copied to clipboard
| Challenge: | Existing studies have generated nods from the final morphemes at the end of an utterance. |
| Approach: | They propose to generate head nods from Japanese dialogues using morphemes . they compile a corpus of 24 dialogues including utterance and nod information . |
| Outcome: | The proposed model outperforms a model using morpheme information in the Japanese language and shows that dialog acts can predict nods. |
Speaker-Aware Discourse Parsing on Multi-Party Dialogues (2022.coling-1)
Copied to clipboard
| Challenge: | Discourse parsing on multi-party dialogues is an important but difficult task in dialogue systems and conversational analysis. |
| Approach: | They propose a speaker-aware model for parsing on multi-party dialogues using interaction features between different speakers. |
| Outcome: | The proposed model achieves the best-reported performance on two standard benchmark datasets. |
Reconstructing Capsule Networks for Zero-shot Intent Classification (D19-1)
Copied to clipboard
| Challenge: | Existing methods for intent classification are limited due to fast-emerging intents . a recent study shows that existing methods are not effective in recognizing unseen intents. |
| Approach: | They propose to reconstruct capsule networks for zero-shot intent classification by using latent information from labeled utterances. |
| Outcome: | The proposed method outperforms existing methods on two task-oriented dialogue datasets in different languages. |
Automatic Evaluate Dialogue Appropriateness by Using Dialogue Act (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations of dialogue quality rely on human judgments, which are time-consuming, labor-intensive, prone to biases, and lacking objectivity. |
| Approach: | They propose a method that utilizes the underlying patterns of dialogue act transitions to evaluate the appropriateness of chatbot responses. |
| Outcome: | The proposed method proves that human judgments are time-consuming, labor-intensive, and lacking objectivity. |
Automatically Identifying Complaints in Social Media (P19-1)
Copied to clipboard
| Challenge: | Complaining is a basic speech act used to express a negative mismatch between reality and expectations in a particular situation. |
| Approach: | They present a systematic analysis of complaints in computational linguistics . they collect annotated data set of written complaints expressed on Twitter . |
| Outcome: | The proposed model achieves predictive performance of up to 79 F1 using distant supervision. |
Data-Driven Pronunciation Modeling of Swiss German Dialectal Speech for Automatic Speech Recognition (L18-1)
Copied to clipboard
| Challenge: | a Swiss German speech recognizer is trained using a standard German annotation model. |
| Approach: | They propose to train a Swiss German speech recognition system using a standard German annotation model. |
| Outcome: | The proposed system is based on a standard German annotation model and a grapheme-to-phoneme conversion model. |
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can produce fluent dialogue but often hallucinate factual inaccuracies. |
| Approach: | They propose a modular model for incorporating knowledge into conversational agents that generates a knowledge sequence and then attends to its own generated knowledge sequence. |
| Outcome: | The proposed model hallucinates less in knowledge-grounded dialogue tasks and has advantages in terms of interpretability and modularity. |
Dual Hierarchical Dialogue Policy Learning for Legal Inquisitive Conversational Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for conversational AI are user-driven, but in many real-world situations, they do not extract information to achieve its own objectives. |
| Approach: | They propose an inquisitive conversational agent that learns when and how to ask probing questions . they also propose a framework for a conversational ICA specifically tailored to the court . |
| Outcome: | The proposed method outperforms single-agent RL baselines on a U.S. Supreme Court dataset. |
MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback (2025.naacl-long)
Copied to clipboard
| Challenge: | Generating multiple-choice questions (MCQG) for professional exams is challenging due to outdated knowledge, hallucination issues, and prompt sensitivity. |
| Approach: | They propose a framework for converting medical cases into high-quality USMLE-style questions using a self-refine-based framework. |
| Outcome: | The proposed framework improves human expert satisfaction regarding quality and difficulty of medical questions. |
Ordinal and Attribute Aware Response Generation in a Multimodal Dialogue System (P19-1)
Copied to clipboard
| Challenge: | Existing multimodal dialogue systems are based on unimodal sources, capturing information from text and image. |
| Approach: | They propose a position and attribute aware attention mechanism to learn enhanced image representation conditioned on the user utterance. |
| Outcome: | The proposed model outperforms the state-of-the-art models on text similarity metrics. |
From Pixels to Personas: Investigating and Modeling Self-Anthropomorphism in Human-Robot Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that robots display human-like characteristics in dialogues . this anthropomorphism raises concerns about the accuracy of AI and its capabilities . |
| Approach: | They propose to use a dataset to analyze self-anthropomorphic and non-self-anthropophilic responses in robots . they propose to combine these two types of responses to create a new category of bot responses . |
| Outcome: | The proposed approach preserves the original dialogues from existing corpora and enhances them with paired responses: self-anthropomorphic and non-self-anthropophilic for each original bot response. |
Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluation of dialog systems are expensive and not scalable . a framework for estimating human evaluation scores is proposed to bridge this gap . |
| Approach: | They propose a framework for estimating human evaluation scores based on off-policy evaluation . they use language quality metrics for single-turn response generation given a fixed context . |
| Outcome: | The proposed framework outperforms existing methods in terms of correlation with human evaluation scores. |
Exploring the Impact of Human Evaluator Group on Chat-Oriented Dialogue Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Evaluator groups such as domain experts, university students, and crowdworkers have been used to assess and compare chat-oriented dialogue systems. |
| Approach: | They analyze the impact of evaluator groups on dialogue system evaluation by testing 4 state-of-the-art dialogue systems using 4 distinct evaluer groups. |
| Outcome: | The proposed evaluations show that the evaluator group impact is not seen for Pairwise, and that it is beneficial for certain metrics. |
NatCS: Eliciting Natural Customer Support Dialogues (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue datasets do not reflect the expected characteristics of real customer support conversations. |
| Approach: | They propose to collect real customer service conversations from real conversations . they show that dialogue act annotations provide more effective training data . |
| Outcome: | The proposed approach is more representative of real human-to-human conversations compared to existing dialogue datasets . the proposed approach can be used to facilitate open research in natural dialog systems . |
Dialogue Collection for Recording the Process of Building Common Ground in a Collaborative Task (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on the process of building common ground have not been well conducted. |
| Approach: | They propose a method for recording the process of building common ground through a dialogue by using the intermediate result of a task. |
| Outcome: | The proposed method can record the building common ground process by using the intermediate result of a task and can be estimated quite accurately. |
MMChat: Multi-Modal Chat Dataset on Social Media (2022.lrec-1)
Copied to clipboard
| Challenge: | Incorporating multi-modal contexts in conversation is important for developing engaging dialogue systems. |
| Approach: | They propose a large scale Chinese multi-modal dialogue corpus that contains image-grounded dialogues from real conversations on social media. |
| Outcome: | The proposed model can handle sparsity issues in dialogue generation tasks by incorporating image features. |
Creating Large-Scale Argumentation Structures for Dialogue Systems (L18-1)
Copied to clipboard
Kazuki Sakai, Akari Inago, Ryuichiro Higashinaka, Yuichiro Yoshikawa, Hiroshi Ishiguro, Junji Tomita
| Challenge: | Argumentation is a process of reaching consensus through premises and rebuttals and is important for making decisions and exchanging views. |
| Approach: | They propose to create argumentation structures in ten languages using argumentation databases . they also examine differences between the two languages to determine their effectiveness . |
| Outcome: | The proposed arguments can be applied to argumentative dialogue systems and can be used as training data. |
PCMID: Multi-Intent Detection through Supervised Prototypical Contrastive Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to intent detection assume that each utterance represents only a single intent. |
| Approach: | They propose a framework for intent detection that can learn multiple representations of a given user utterance under the context of different intent labels in an optimized semantic space. |
| Outcome: | The proposed framework achieves state-of-the-art on multiple public benchmark datasets and a private real-world dataset for the multi-intent detection task. |
Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve accuracy of neural networks are slow due to computational complexity. |
| Approach: | They propose a vector-vector-matrix architecture which greatly reduces latency at inference time for NLP applications by a factor of four. |
| Outcome: | The proposed framework reduces the latency of sequence-to-sequence and Transformer models used for NMT by a factor of four. |
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models enable them to follow freeform instructions, including imitating generic or specific demographic personas in conversations. |
| Approach: | They propose to investigate persona biases by experimenting with UNIVERSALPERSONA, a model that incorporates both generic and specific personas. |
| Outcome: | The proposed model systematically measures persona biases in harmful expression and harmful agreement. |
Mixed-Session Conversation with Egocentric Memory (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent dialogue systems exhibit an inability to replicate dynamic, continuous, long-term interactions involving multiple partners. |
| Approach: | They propose a multi-session dialogue system that builds on real-world interactions by integrating deep layered interactions and widening conversation networks. |
| Outcome: | The proposed system is based on a dataset of 6 consecutive dialogue episodes with four speakers (one main speaker and three partners) appearing in each episode. |
RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing intent detection approaches have relied on adaptively pre-training language models with large-scale datasets, yet the predominant cost of data collection may hinder their superiority. |
| Approach: | They propose a self-supervised framework dedicated to task-oriented dialogues which incorporates agent responses for pre-training in a two-stage manner. |
| Outcome: | The proposed framework outperforms the state-of-the-art frameworks for task-oriented dialogues on two real-world customer service datasets. |
Visualizing Dialogues: Enhancing Image Selection through Dialogue Understanding with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for dialogue-to-image retrieval are constrained by pre-trained vision language models. |
| Approach: | They leverage the reasoning capabilities of large language models to predict potential features in images to be shared based on dialogue context. |
| Outcome: | The proposed method outperforms existing methods significantly in terms of Recall@k. |
SlugNERDS: A Named Entity Recognition Tool for Open Domain Dialogue Systems (L18-1)
Copied to clipboard
| Challenge: | UCSC researchers have developed an open domain social bot aimed at casual conversation . NER and NEL are important preprocessing steps for understanding user intent in open domain dialogue systems. |
| Approach: | They propose a tool for NER and NEL in open domain dialogue that addresses these challenges . they also propose two corpora based on 10,000 real user conversations . |
| Outcome: | The proposed open domain social bot is aimed at casual conversation. |
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation (2025.acl-long)
Copied to clipboard
Qinglin Zhang, Luyao Cheng, Chong Deng, Qian Chen, Wen Wang, Siqi Zheng, Jiaqing Liu, Hai Yu, Chao-Hong Tan, Zhihao Du, ShiLiang Zhang
| Challenge: | Full-duplex spoken dialogue systems allow simultaneous bidirectional communication . low latency and natural interactions in full-duplice systems remains a challenge . |
| Approach: | They propose a multi-stage post-training scheme that adapts a text large language model into a speech-text dialogue LLM. |
| Outcome: | The proposed model can model human conversation behaviors with low latency and natural interactions with low delay. |
Mind the Gap Between Conversations for Improved Long-Term Dialogue Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a gap between conversations can be weeks, months or years, and dialogue systems which do not explicitly model time may generate unnatural responses. |
| Approach: | They propose to model the passage of time between conversations by exposing time information to a multi-session dialogue dataset and comparing different representations of time and event progress. |
| Outcome: | The proposed model is based on a real-time dataset showing that it can predict topics and information gained from conversations over a long time span. |
ESCoT: Towards Interpretable Emotional Support Dialogue Systems (2024.acl-long)
Copied to clipboard
| Challenge: | Emotion-focused and strategy-driven chain-of-thought (ESCoT) is a new paradigm for emotional support dialogues. |
| Approach: | They propose an emotional support response generation scheme to improve interpretability . they generate a dataset and develop a model to generate dialogue responses with better interpretability. |
| Outcome: | The proposed scheme can generate dialogue responses with better interpretability. |
Disfluency Generation for More Robust Dialogue Systems (2023.findings-acl)
Copied to clipboard
| Challenge: | Disfluencies in user utterances can trigger a chain of errors impacting all the modules of a dialogue system. |
| Approach: | They propose to augment existing dialogue datasets with disfluent utterances by paraphrasing them into disfluente ones. |
| Outcome: | The proposed method improves dialogue state tracking and response generation by combining disfluent utterances with disfluency utteraces. |
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts (2024.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for the automated evaluation of natural language texts are based on a large language model (LLM) that fails to agree with human judges and is not fully validated by the human judges. |
| Approach: | They propose a large language model (LLM) that generates a distribution over potential responses to assess multiple dimensions of interest. |
| Outcome: | The proposed framework predicts human judges' assessment of user satisfaction on a scale of 1–4 with an RMS error 0.5, a 2 improvement over the uncalibrated baseline. |
ScriptWriter: Narrative-Guided Script Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing systems that generate scripts from a storyline are not available for this purpose. |
| Approach: | They propose a model that generates a story from a narrative and a tool that keeps track of what is said and what is to be said. |
| Outcome: | The proposed model outperforms baselines that use the narrative as a kind of context. |
Syntax-driven Approach for Semantic Role Labeling (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies focus on auto-generated syntactic knowledge to enhance semantic role labeling . experimental results show that map memories can enhance SRL . |
| Approach: | They propose to map memories to enhance semantic role labeling by encoding auto-generated syntactic knowledge from off-the-shelf toolkits. |
| Outcome: | The proposed model outperforms baselines and achieves state-of-the-art results on two English benchmark datasets. |
Multimodal Persona Based Generation of Comic Dialogs (2023.acl-long)
Copied to clipboard
| Challenge: | Existing models for persona based dialogue generation for comic strips encode two-party dialogues and do not account for visual information. |
| Approach: | They propose a multimodal persona-based architecture to generate dialogues for the next panel in comic strips. |
| Outcome: | The proposed paradigm reduces the perplexity score by 10 points over existing models . the novel dataset, ComSet, contains 54K comic strips . |
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs generate responses based on the dialogue context, overlooking the underlying linguistic cues about the user status exhibited in the context. |
| Approach: | They propose a linguistic cue-based chain-of-thoughts method which enhances the LLMs inference with an intermediate reasoning step to find cues exhibited in the dialogue. |
| Outcome: | The proposed method outperforms standard prompting methods on in-depth dialogue questions and linguistic cues exhibited in the context. |
Don’t Forget Your ABC’s: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems (2023.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods are biased because of their subjectivity and inconsistent evaluation can misinform the performance of a chat-oriented open-domain dialogue system. |
| Approach: | They propose to use a human evaluation method to estimate the rates of manypasted macro ‘LN’ dialogue system behaviors to compare them with existing evaluation methods. |
| Outcome: | The proposed method is more suitable than alternative Likert-style or comparative approaches for dimensional evaluation of open-domain dialogue systems. |
Local and Global Decoding in Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Text generation relies heavily on decoding algorithms that sample strings from a language model distribution. |
| Approach: | They propose to introduce globally-normalised versions of traditional decoding methods and propose an independent Metropolis-Hastings algorithm to approximate sampling from globally-averaged distributions without explicitly computing them. |
| Outcome: | The proposed method approximates the distributions without explicitly computing them. |
IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional data generation methods are labor-intensive, resource-demanding, and raise privacy concerns. |
| Approach: | They propose an automatic synthetic data generation approach and introduce the **I**mplicit **P**ersonalized **Dialog**ue benchmark along with a training dataset, covering 10 tasks and 12 user attribute types. |
| Outcome: | The proposed approach incorporates the **Implicit **P**ersonalized **Dialog**ue benchmark along with a training dataset, covering 10 tasks and 12 user attribute types. |
Evaluating the Long-Term Memory of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have not thoroughly investigated the memory performance of large language models in long-term tasks. |
| Approach: | They propose a dataset to evaluate the long-term memory capabilities of large language models. |
| Outcome: | The proposed model exhibits memory preferences across different categories of information. |
Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles (2025.acl-long)
Copied to clipboard
| Challenge: | Existing user simulators lack authenticity and user-level diversity in interactions with large language models. |
| Approach: | They propose a user simulator with implicit user profiles that infers user profiles from human-machine interactions to simulate personalized and realistic dialogues. |
| Outcome: | The proposed framework outperforms baselines in authenticity and diversity while maintaining comparable consistency. |
FFAEval: Evaluating Dialogue System via Free-For-All Ranking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation metrics for open-domain dialogue systems show poor correlation with human assessment. |
| Approach: | They propose a free-for-all human evaluation framework that shares dialogue history with annotators for multi-turn scoring. |
| Outcome: | The proposed framework achieves a strong correlation with human assessment on English and Chinese dialogue systems. |
nEMO: Dataset of Emotional Speech in Polish (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets covering Slavic languages do not accurately represent basic emotional states. |
| Approach: | They propose to use a Polish corpus of emotional speech to represent basic emotional states. |
| Outcome: | The proposed corpus represents six emotional states in Polish, with 9 actors participating in the study. |
Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for OOD intent detection are limited to single dialogue turns. |
| Approach: | They propose a context-aware OOD intent detection framework to model multi-turn contexts in OOD context detection tasks using unlabeled data. |
| Outcome: | The proposed framework improves the F1-OOD score by 29% on multi-turn OOD detection tasks compared to the previous best method. |
Know Your Place: Diagnosing Implicit Social Adaptation Failures in Chinese Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies suggest that failures of large language models in social contexts are not due to limited linguistic competence, but to inappropriate recognition. |
| Approach: | They propose a framework that decomposes social adaptation into three orthogonal dimensions and conduct controlled comparisons across multiple Chinese LLMs under implicit and explicit conditions. |
| Outcome: | The proposed framework decomposes social adaptation into three orthogonal dimensions and conducts controlled comparisons across multiple Chinese LLMs under implicit and explicit conditions. |
LongMP-Bench: A Benchmark for Multimodal Persona Understanding in Long-Term Dialogues (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets suffer from limited persona diversity and static, overly simplified settings, making them insufficient for capturing the complexity of real-world interactions. |
| Approach: | They propose a benchmark to evaluate models' ability to understand evolving user personas within long-term multimodal dialogues by using a dataset that contains long conversations from 150 users. |
| Outcome: | The proposed benchmark aims to assess models' ability to track persona evolution, integrate visual and textual inputs, and apply persona understanding in realistic dialogue scenarios. |
Reimagining Intent Prediction: Insights from Graph-Based Dialogue Modeling and Sentence Encoders (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to intent prediction are limited in highly specialized fields, such as closed-domain dialogue systems, where context comprehension is of paramount importance. |
| Approach: | They propose a method that uses scenario dialog graphs to model dialogues as sequences of transitions between intents, representing distinct goals or requests. |
| Outcome: | The proposed method significantly advances the field of dialogue systems, providing valuable insights into the effectiveness and potential limitations of the proposed approaches. |
Follow-up Question Generation For Enhanced Patient-Provider Conversations (2025.acl-long)
Copied to clipboard
Joseph Gatto, Parker Seegmiller, Timothy E. Burdick, Inas S. Khayal, Sarah DeLozier, Sarah Masud Preum
| Challenge: | Follow-up question generation is an essential feature of dialogue systems as it can reduce conversational ambiguity and enhance modeling complex interactions. |
| Approach: | They propose a framework that generates personalized follow-up questions based on patient utterances and prior EHR data. |
| Outcome: | The framework reduces follow-up communications by 34% and improves performance by 17% and 5% on real and synthetic data. |
When Words Smile: Generating Diverse Emotional Facial Expressions from Text (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems that generate only coarse facial expressions ignore the rich and dynamic nature of face-to-face communication. |
| Approach: | They propose an end-to-end text-to expression model that explicitly focuses on emotional dynamics. |
| Outcome: | The proposed model outperforms baselines on 15,000 text–3D expression pairs on a large-scale dataset. |
CogEmp:A Cognitive Empathy-Oriented Dialogue System for Structured Psychological Counseling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models lack accurate modeling of cognitive empathy, especially the ability to understand users’ emotions and their underlying psychological causes. |
| Approach: | They propose a model tailored for the Chinese cultural context that integrates cognitive empathy into LLMs. |
| Outcome: | The proposed model outperforms existing models in key evaluation metrics, particularly in empathy, comprehensibility, and professionalism. |
NormGenesis: Multicultural Dialogue Generation via Exemplar-Guided Social Norm Modeling and Violation Recovery (2025.emnlp-main)
Copied to clipboard
| Challenge: | Social norms govern culturally appropriate behavior in communication, enabling dialogue systems to produce coherent and socially acceptable outputs. |
| Approach: | They propose a framework for generating and annotating socially grounded dialogues in Chinese, English, and Korean. |
| Outcome: | The proposed framework outperforms existing frameworks in refinement quality, dialogue naturalness, and generalization performance. |
VISTA: Verification In Sequential Turn-based Assessment (2026.acl-long)
Copied to clipboard
| Challenge: | Existing metrics evaluate isolated responses or treat unverifiable content as errors, limiting their use for multi-turn dialogue. |
| Approach: | They propose a framework for evaluating conversational factuality via claim-level verification and sequential consistency tracking. |
| Outcome: | The proposed framework improves hallucination detection over existing benchmarks and models. |
Schema-Guided Response Generation using Multi-Frame Dialogue State for Motivational Interviewing Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Motivational interviewing (MI) is a goal-directed dialogue aimed at motivating clients to change their behavior. |
| Approach: | They propose a method for updating multi-frame dialogue states and a strategy decision mechanism that dynamically determines the response focus in a manner grounded in MI principles. |
| Outcome: | The proposed method generates responses aligned with MI principles and frequently asks questions to elicit change talk. |